The Physics of Information Information Theory in the Light of Thermodynamics, Statistical Mechanics and Nonlinear Dynamics

1. The Physics of Information 2 2. Thermodynamics 2 2.1. The laws 3 2.2. Free energy 6 3. Statistical mechanics 7 3.1. Definitions and postulates 8 3.2. A simple model system of magnetic spins 9 3.3. The Maxwell-Boltzmann distribution 10 3.4. Free energy revisited 11 3.5. Gibbs entropy 12 4. Nonlinear dynamics 12 4.1. The ergodic hypothesis 13 4.2. Chaos and limits to prediction 14 4.3. Quantifying predictability 16 5. About Entropy 19 5.1. Entropy and information 19 5.2. The entropy as a relative concept 20 5.3. The Gibbs paradox 21 5.4. Adding the entropy of subsystems 22 5.5. The maximal entropy principle of Jaynes 24 5.6. Ockham’s razor 25 5.7. Coarse graining and irreversibility 25 5.8. Coarse graining and renormalization 28 5.9. Beyond the Boltzmann, Gibbs and Shannon entropy: the Tsallis entropy 29 6. Black Holes: a space time information paradox 31 7. Information in design and engineering 33 8. Conclusion 36 References 36